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RESULTS  FROM  16/NOVEMBER/2000  -  16/FEBRUARY/2000 


Introduction 

This  is  the  final  report  that  describes  the  option  period  for  this  contract.  During  this  option,  we  are  being  asked  to 
participate  in  a  competition  between  various  vendors  to  generate  the  best  digital  target  map  for  the  proposed  sensor. 
To  the  best  of  our  knowledge,  this  sensor  is  a  coherent  radar  with  improved  range  resolution  that  enables  it  to  detect 
fine  elevation  differences  in  the  terrain  and  to  compare  them  with  a  stored  elevation  signature  of  the  area  upon  target 
approach.  Although  our  algorithms  are  in  their  infancy,  we  accepted  the  challenge  and  will  attempt  to  produce  an 
appropriate  elevation  signature  description  using  the  data  supplied. 


Processing  Results  Description 

On  the  CD,  you  will  find  two  tools  entitled  Stereo  Viewer  and  Mono  Viewer.  There  usage  is  described  in  Appendix  A 
&  B,  respectively.  They  allow  the  operator  to  view  the  image  with  the  overlay  of  the  feature  processing  results.  The 
image  can  be  viewed  and  the  result  of  the  process  will  be  overlaid  in  ’’red”  and  aligned  with  the  image  pixels  that  it 
matched.  The  image  is  a  backdrop  to  demonstrate  the  process  and  describe  its  accuracy  and  time  efficiency 
compared  to  traditional  correlation  approaches. 

The  time  to  generate  this  result  shown  and  the  accuracy  of  the  detections  were  key  considerations  in  the 
development  of  our  approach.  An  important  fact  is  that  the  sensor’s  image  processor  is  sensitive  to  the  data  that  it  is 
asked  to  correlate  upon.  The  feature  result  shown  is  consistent  with  the  sensor’s  image  map  and  will  produce  an 
optimal  correlation  from  which  the  inertial  navigation  system  can  be  updated. 

The  analysis  performed  on  the  imagery  may  create  results  in  which  too  few  detections  or  an  inordinately  large 
number  of  detections  would  be  generated  to  pass  on  to  the  matching  process.  The  result  shown  on  the  CD  resulted 
from  choosing  thresholds  in  the  pre-processing  stage  of  the  algorithm  that  resulted  in  an  optimized  set  of  matching 
points  that  rely  upon  strong  edge  boundaries  within  the  image.  It  has  been  shown  in  several  studies  that  the  strong 
edges  in  an  optical  scene  are,  for  the  most  part,  also  the  strong  edges  in  an  IR  or  SAR  scene. 
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If  a  regular  grid  is  generated  by  interpolating  the  results  shown,  an  inaccurate  representation  of  the  dominant  targets 
in  the  scene  will  result.  This  generates  a  non-optimal  match  when  the  seeker  attempts  to  correlate  its  detections 
against  the  grid  presented  to  it.  Also,  each  of  these  points  generated  by  interpolation  must  be  matched  within  the 
targeting  seeker’s  image.  This  increases  the  processing  requirements  of  the  seeker  processor  and  can  degrade  the 
overall  match  process  by  adding  detections  that  must  be  equally  weighted  in  the  final  navigation  update  for  the 
targeting  sensor  while  contributing  little  to  the  uniqueness  of  the  match  itself 

Each  of  these  correlation  results  represent  the  x,  y  pixel  location  of  the  image  pixel  within  the  left  and  right  stereo 
pair  for  a  given  feature  such  as  a  building  rooftop  or  road  boundary.  By  using  the  image  -  to  -  geodetic 
transformation  (rational  polynomial  coefficient  application  in  the  case  of  DPPDB),  the  geodetic  location  (X,  Y,  &  Z) 
can  be  provided  to  the  targeting  sensor  with  the  full  accuracy  of  the  product  from  which  it  was  extracted  (DPPDB 
product  accuracy  if  this  source  imagery  is  used).  The  process  results  in  an  optimized  version  of  a  Digital  Elevation 
Model  or  DEM.  The  number  of  detections  from  which  a  match  pixel  can  be  generated  is  controlled  by  the  Canny 
edge  detection  threshold  logic.  This  process  is  also  extremely  efficient  computationally  since  only  those  detections 
that  were  detected  as  strong  edge  detections  are  used  in  the  matching  process.  We  have  also  optimized  both  the 
Canny  edge  detection  pre-processing  and  the  correlation  match  geodetic  feature  extraction  algorithms. 

There  are  edges  that  are  generated  by  shadows  in  the  optical  scene  that  may  or  may  not  exist  in  the  targeting  image; 
however,  this  fact  is  irrelevant  because  each  of  the  detections  that  are  shown  in  the  map  have  an  accurate  X,  Y,  Z 
geodetic  coordinate  attached  to  them.  They  represent  a  physical  geodetic  location  of  the  shadow  on  the  ground. 


Test  Results 

A  series  of  tools  were  created  to  exploit  the  DPPDB  product  and  display  the  processing  results.  A  subset  of  this 
toolset  enables  the  viewing  and  exploitation  of  the  correlation  results  that  were  generated.  The  toolset  allows  for 
investigation  of  various  tuning  parameters  and  settings  to  provide  insight  into  the  appropriate  threshold  values 
necessary  to  extract  the  desired  features.  All  of  the  results  from  this  imagery  and  subsequent  test  imagery  are  being 
supplied  on  a  classified  CD  as  an  addendum  to  this  final  report. 


3 


After  the  imagery  is  extracted  from  the  DPPDB  segment,  a  preprocessing  stage  is  performed  on  the  imagery.  This 
creates  a  template  description  of  the  pixels  in  the  left  stereo-pair  image  from  which  to  drive  the  correlation 
processing  in  the  right  stereo-pair  image.  On  the  CD  are  all  of  the  original  image  chips  in  TIFF  format.  With  the 
Stereo  Viewer  or  Mono  Viewer  tool  supplied  on  the  CD,  the  user  can  investigate  the  performance  of  the  algorithm 
and  tune  its  results  to  generate  the  desired  DEM  for  a  particular  sensor  scenario. 

The  correlation  processing  looks  in  the  right  stereo-pair  for  corresponding  matches  to  the  template  description  in  the 
stereo-pair.  The  current  state  of  algorithm  development  does  not  always  provide  a  one-to-one  match.  The  possible 
results  include  finding  a  correct  match,  finding  no  match,  and  finding  an  incorrect  match.  Since  we  did  not  have 
access  to  the  ‘truth’  data  prior  to  this  report,  an  accuracy  assessment  could  only  be  conducted  on  a  qualitative  basis. 
For  each  area  tested  three  sample  areas  were  chosen  to  evaluate  the  algorithms  performance.  Two  of  the  areas  were 
representative  of  the  entire  scene  results  and  the  third  area  was  selected  as  a  worse  case  scenario.  Whenever  a  match 
in  the  right  stereo-pair  correctly  corresponds  to  the  left  stereo-pair  template,  the  algorithm  returns  the  position  at  the 
accuracy  inherent  to  DPPDB.  The  next  step  in  algorithm  development  will  be  to  add  a  contextual  processing  step 
that  will  identify  additional  parameters  in  the  left  stereo-pair  template  that  will  be  used  to  drive  the  correlation¬ 
processing  algorithm. 


The  following  test  sites  and  their  associated  statistics  are  shown  in  Table  I. 


Source  area 

Terrain/Culture 

Type 

Processing  Time 

Probability  of  success 

Total  points/matches 

Comments 

Baltimore, MD 

High  density  urban 

6  minutes 

1)  157/153  97.5% 

2)  140/140  100% 

3)  193/173  89.6% 

The  algorithm  results 

were  excellent  for  this 

cloud  covered  scene 

San  Francisco, 

CA 

High  density  urban 

7  min 

1)  147/144  98.0% 

2)  181/179  98.9% 

3)  133/100  75.2% 

The  algorithm 

performed  well  with  the 

convergent  stereo  data 

within  this  imagery 
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Fort  Polk,  LA 

Tree  lines/no  relief 

6  minutes 

1)  131/131  100% 

2)  143/143  100% 

3)  160/160  100% 

This  was  a  simple  scene 

for  this  algorithm 

Meling 

Orchard,  CA 

repetitive  patterns 

6  minutes 

1)  163/163  100% 

2)  198/198  100% 

3)  202/183  90.6% 

Another  simple  scene 

Fort  Benning, 

GA 

Hills,  gullies  and 

airfield 

7  minutes 

1)  142/142  100% 

2)  126/126  100% 

3)  116/70  60.3% 

Highly  correlated  terrain 

contours  with 

shadowing  in  third 

sample 

Oakland,  CA 

High  density  urban 

6  minutes 

1)  178/173  97.2% 

2)  160/160  100% 

4)  195/121  63.1% 

Highly  correlated 

building  features  with 

shadowing  in  third 

sample 

An  interesting  result  occurs  if  you  load  the  left  stereo-pair  edge  points  into  MonoViewer  and  then  load  the  DEM 
post  results  into  the  same  window.  The  result  is  shown  in  figure  1  and  this  gives  an  indication  of  the  image  positions 
within  the  image  that  mis-matched  dining  the  correlation  process.  The  red  pixels  represent  the  points  within  the 
image  that  correctly  matched  to  create  stereo  DEM  posts.  The  black  pixels  are  the  edges  that  were  detected  in  the 
left  image;  but  which  did  not  have  a  corresponding  match  in  the  right  image. 
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Figure  1:  Edge  processing  result  overlaid  with  the  “red”  correlation  match  result 


In  closing,  it  should  be  noted  that  this  entire  process  is  automated.  After  the  thresholds  are  determined,  the  entire 
DEM  was  generated  by  the  algorithms  and  stored  on  the  computer.  No  human  interaction  was  performed  during  its 
creation. 

PROJECT  STATUS 

No  impact  or  significant  change  to  the  planned  work  schedule  has  occurred. 


RECOMMENDATIONS  AND  PROPOSALS 

None. 
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APPENDIX  A  -  StereoViewer  Application  Notes 


Background 

On  the  CD,  you  will  find  a  tool  entitled  “Stereo Viewer”.  This  tool  requires  special  graphics  hardware  and  shuttered 
stereo-viewing  glasses  to  enable  the  view  to  appear  stereoscopic.  It  allows  you  to  view  the  image  with  the  overlay  of 
the  DEM  feature  processing  results.  The  image  can  be  viewed  and  the  result  of  the  process  will  be  overlaid  in  ’Ted” 
and  aligned  with  the  image  pixels  that  it  matched. 


Using  Stereo  Viewer 

To  use  the  StereoViewer  program  that  is  supplied  on  the  CD,  open  the  “Stereoviewer.exe”  file  on  the  CD  by 
“double-clicking”  on  it.  The  image  window  will  appear  as  shown  in  figure  1.  The  3  sets  of  “arrow”  keys  are  used  to 
shift  within  the  left  image,  right  image,  or  both  images  in  the  pixel  increments  displayed.  The  “magnification” 
button  will  magnifiy  or  minify  the  contents  of  the  viewing  window.  The  first  set  of  scroll  arrows  allows  the  user  to 
shift  the  left  image  by  an  incremental  amount  shown  in  the  text  button  box  next  to  it.  Similarly,  the  second  set  of 
arrows  will  have  the  effect  of  shifting  the  right  image  relative  to  the  left.  Either  of  these  sets  of  arrow  buttons  allows 
the  user  to  adjust  the  stereo  baseline  to  enable  the  proper  viewing  of  objects  in  the  stereo  scene  with  varying 
amounts  of  x-parallax.  Since  the  stereo  model  is  scaled  to  accommodate  the  viewing  physics  of  a  human’s  binocular 
vision  system,  these  adjustments  are  necessary.  The  3rd  set  of  arrow  buttons  is  used  to  scroll  within  the  stereo 
imagery  and  does  not  affect  the  parallax  but  allows  the  user  to  view  different  areas  of  the  image  on  the  screen. 
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Figure  1 :  Main  viewer  window 


On  the  “file”  menu  use  the  file  dialogue  box  of  figure  2  will  appear  when  “Open  images...”  is  selected.  Double¬ 
click  on  the  “left.tif5  file  icon  and  then  “double-click”  on  the  “right. tif’  image  icon.  It  will  take  a  few  seconds  to 
display  both  images  in  the  main  viewing  window. 
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Figure  2:  file-open  dialogue  box 

Return  to  the  “file”  menu  and  select  “Open  DEM...”  from  the  file  menu.  The  file  dialogue  box  of  figure  3  will 
appear  and  select  the  “P21_S50_T50.dem”  file  by  “double-clicking”  on  its  icon. 


Figure  3 :  Feature  dialogue  box  for  the  DEM 

In  Figure  4,  the  “red”  shaded  overlaid  pixels  represent  the  final  result  of  the  Canny  preprocessing  and  Correlation 
matching  operation.  Each  of  these  “red”  pixels  represent  an  object  pixel  in  the  left  and  right  stereo-pair  from  which  a 
highly  accurate  geo-location  position  (X,  Y,  &  Z)  exists. 
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Figure  4:  Image  overlaid  with  “red”  correlation  detections 
For  best  viewing  results,  set  the  “magnification”  to  50%  and  enlarge  the  Stereoview  window  to  full-screen  by 
“clicking”  on  the  2nd  box  button  on  the  upper-right  of  the  Stereoview  window  -  this  is  like  enlarging  any  window  on 
a  PC  to  full-screen.  With  the  stereo-glasses  on,  scroll  through  the  image  by  using  the  3rd  set  of  scroll  arrows  from  the 
left-side  of  the  main  window  in  the  tool,  this  scrolls  both  the  left  and  right  images  of  the  stereo-pair  in  unison 
ensuring  that  the  parallax  does  not  change. 

The  geo-coded  location  of  the  current  cursor  is  shown  at  the  bottom  of  the  window  in  real-time.  If  you  desire  to 
measure  the  height  of  a  building  or  structure,  scroll  to  it  with  the  cursor  and  use  the  “middle”  mouse  button  to  lower 
the  cursor  and  the  “right”  mouse  button  to  raise  the  cursor  to  the  proper  height  of  the  building.  The  height  value  at 
the  bottom  of  the  screen  will  represent  the  current  height  of  the  cursor  and  thus  the  object. 
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APPENDIX  B  -  MonoViewer  Application  Notes 

Using  MonoViewer 

To  use  the  MonoViewer  program  that  is  supplied  on  the  CD,  open  the  “Monoviewer.exe”  file  on  the  CD  by  “double¬ 
clicking”  on  it.  The  image  window  will  appear  as  shown  in  figure  1.  The  “arrow”  keys  are  used  to  scroll  within  the 
image  in  pixel  increments.  To  change  the  scroll  increment,  “click”  on  the  “scroll-increment”  button  and  select  from 
one  of  the  values  (i.e.  1,  10,  100,  128,  256,  512,  or  1024). 


Figure  1 :  Main  viewer  window 

On  the  “file”  menu,  select  the  “open”  button  and  the  file  dialogue  box  of  figure  2  will  appear.  Double-click  on  the 
“left.tif”  file  icon  and  the  image  will  appear  in  the  main  window. 
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Return  to  the  “file”  menu  and  select  “Open  DEM”  from  the  file  menu.  The  file  dialogue  box  of  figure  3  will  appear 
and  select  the”  left.dem”  file  by  “double-clicking”  on  its  icon. 


JLj2£j 


File  name: 

Files  of  type:  |DEM(’dem)  3  Cancel  I 

A 


Figure  3:  Feature  dialogue  box  for  the  DEM 

In  figure  4,  the  “red”  shaded  overlaid  pixels  represent  the  final  result  of  the  Canny  preprocessing  and  Correlation 
matching  operation.  Each  of  these  “red”  pixels  represent  an  object  pixel  in  the  left  and  right  stereo-pair  from  which  a 
highly  accurate  geo-location  position  (X,  Y,  &  Z)  can  potentially  exist.  Each  of  these  edge  points  is  used  to  direct 
the  feature-based  correlation  algorithm  to  the  area  in  the  right  stereo-pair  from  which  to  initiate  a  search  for  the 
correct  stereo  matching  pixel. 
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Figure  4:  Left  Stereo-pair  image  overlaid  with  “red”  correlation  detections 
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